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Young children and their families benefit from early care and education (ECE) settings that implement high- 
quality practices that support children’s well-being, health, and safety. Activities to support quality are a 
focal point of federal and state investments, including more than $1 billion of the Child Care and 
Development Fund and nearly $240 million in Head Start Training and Technical Assistance .* Quality 
improvement activities include avariety of delivery modes and content areas to address children’s 
developmental needs and to align with the knowledge and skills of the ECE workforce. The work of quality 
improvement in ECE is challenging using existing methods, and sustained changes in quality across ECE 
settings have not been widespread.2 ECE leaders and staff are seeking new approaches to quality 
improvement that can promote meaningful and sustained changes in practice. The purpose of this brief is to 
share findings from the Culture of Continuous Learning (CCL) project, a study of the Breakthrough Series 
Collaborative (BSC) in ECE programs. The BSC is a quality improvement methodology tested in ECE 
programs to support practice change by addressing individual and organizational factors associated with 
sustaining high-quality practices. 


This brief is structured in the following four sections: 


e Outlining the context for quality improvement in ECE settings and the growing interest in strategies 
that promote ongoing learning and change rather than demonstrating quality at one point in time. 

e Describing the BSC and providing an overview of how the method works to promote change. 

e Sharing what we learned about the ECE programs that participated in the pilot study and if the 
quality improvement method worked as expected. 

e Reflecting on the implications of the findings for quality improvement initiatives in ECE systems. 


This brief will provide information for decision makers at the federal, state, and local levels who fund quality 
improvement strategies. 


The Culture of Continuous Learning Project 


The Culture of Continuous Learning (CCL) Project: A Breakthrough Series Collaborative for Improving Child Care and 
Head Start Quality. The CCL Project is being conducted by Child Trends, the University of Massachusetts-Boston, JRA 
Consulting Ltd., and the Center for the Study of Social Policy. It is funded by the Office of Planning, Research, and 
Evaluation in the Administration for Children and Families in the U.S. Department of Health and Human Services. The 
purpose of the project is to assess the feasibility of implementing a Breakthrough Series Collaborative to support social 
and emotional learning in child care and Head Start programs. The project webpage provides further information and 
context that supplement the content of this brief. 
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Promoting a Culture of Continuous Learning in Early Care and Education Settings 


To preview our key conclusions, the BSC has promise for ECE settings. The method is compatible with a 
focus on continuous improvement promoted by ECE leaders. The method is flexible enough to be used with 
different content areas. Staff in center-based ECE programs participated in the activities and reported 
engaging in the content and the quality improvement process. The research team documented initial 
changes in practices that indicated increased organizational support of quality improvement and movement 
toward amindset of continuous learning. The initial findings also highlight the challenges ECE programs face 
that make it difficult for them to participate in quality improvement. We offer suggestions to decision 
makers interested in implementing quality improvement methods in ECE programs. While this project was 
conducted before the COVID-19 pandemic, the lessons learned provide insight into potential strategies 
being implemented to support recovery of child care and early education programs. 


The Context for Quality Improvementin ECE Programs 


When assessedon traditional measures of observed quality in center-based classrooms, the average scores 
of classrooms in child care centers and Head Start programs are in the middle- to low-ranges of the 
measures.° Teaching practices that support children’s language and concept development score particularly 
low. Improving quality requires understanding the broader context for the ECE workforce and limitations in 
current quality improvement approaches. 


Evidence suggests that compensation and work conditions create economic insecurity and turnover among the 
ECE workforce which may negatively impact the quality of services children and families receive. 


Early educators receive low wages overall, and wage disparities exist among staff of color and staff who are 
white.* Center-based staff earn $12 per hour on average and receive minimal employee benefits.> More 
than half of the child care workforce receives public income supports (such as the Earned Income Tax Credit 
or Supplemental Nutrition Assistance Program).¢ Low wages are associated with increased annual turnover 
in child care programs, estimated at nearly 20 percent of staff in centers.”? Compensation is also associated 
with anational staffing shortage in the wake of the COVID-19 pandemic.® Other conditions impacting the 
child care work environment are related to job stress, burnout, and depressive symptoms.? Across ECE 
settings, compensation and conditions for the workforce negatively affect the stability and quality of 
services provided to children and families.1° Yet, quality improvement methods do not typically address 
compensation, benefits, and other workplace conditions that make it challenging to support the well-being 
of early educators. 


Quality improvement opportunities typically focus on promoting compliance with standards through training 
events or coaching rather than comprehensive approaches to implementing evidence-based practices with the 
unique children and families served in ECE programs. 


Opportunities for ECE programs to participate in meaningful quality improvement and professional 
development activities differ across states, communities, and program types. For example, while most states 
have implemented a quality rating and improvement system (QRIS) for ECE programs, the range and depth 
of quality improvement options available to programs vary across QRIS.11 Nearly all QRIS offer coaching 
and consultation, but the number of hours, format, and content varies across QRIS and across programs 
with different needs. Fewer QRIS offer peer-to-peer learning activities, though use of this format is 
increasing. QRIS typically serve child care centers and family child care programs; fewer Head Start 
programs receive quality improvement supports through a QRIS, evenif they havea rating. 12 Head Start 
programs have access to supports from regional and national technical assistance providers, and a portion 
of each grant is allocated for their own training and technical assistance. Nearly all center directors in Head 
Start report that they offer workshops (sponsored by their own organization or another organization); most 
also offer coaching and consultation.12 Most Head Start teachers (80%) report receiving coaching, with the 
majority receiving coaching at least once per month. Head Start grantee directors are more likely to 
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participate in networks of leaders and leadership institutes than center directors, indicating potential 
discrepancies in how opportunities are offered in Head Start.*4 


Although quality improvement supports vary widely, a commonality among program types, state systems, and 
Head Start grantees is that quality improvement typically focuses on individuals’ skills and competencies. 


Coaching, consultation, and mentoring in QRIS offer strategies to help individual teachers and directors 
meet the quality standards outlined in the QRIS (including scores on observational tools). Less attention and 
limited technical assistance focus on organizational structures and workplace environments that can foster 
and sustain improvements over time.*5A concern among some QRIS leaders is that coaching individuals on 
the standards or observational tools can result in practice changes that are not sustained because they are 
not embedded in the structures and processes of the program. 16 They may also promote point-in-time 
demonstration of skills instead of skills that are part of everyday routines. 


While quality improvement and continuous learning are prioritized and embedded in QRISand the Head Start 
Program Performance Standards, it is challenging for programs to apply these recommendations in everyday 
practices. 


While two-thirds of QRIS offer technical assistance focused on continuous improvement, each QRIS defines 
and implements this focus differently. Continuous improvement may include completing anassessment and 
setting goals based on assessment results but is less likely to include regular data collection and 
improvement cycles to test changes in practices. The 2016 Head Start Program Performance Standards 
require programs to engage in monitoring and continuous improvement, but evidence suggests that Head 
Start programs have progress to make in using data for program improvement.” 


Addressing quality improvement in the current context thus requires approaches that consider the wide 
variety of experiences early educators have with quality improvement and the challenges of low 
compensation, turnover, and staffing shortages. Approaches that acknowledge the importance of 
organizational structures and climate, collaborative approaches, and ongoing learning would be valuable in 
ECE systems. 


Creating a Culture of Continuous Improvement 


Quality improvement methods used in health care and business can be applied to ECE. Research indicates 
their effectiveness and their focus on the content and the process of improvement, including developing an 
infrastructure that supports quality improvement over time.?® Quality improvement methods provide 
content neutral tools that can be applied across different teaching practices in ECE. The structures for 
implementing the tools help build positive dynamics among program leaders and staff. By creating a positive 
work climate and aculture of continuous learning, changes in practice can be sustained over time. 


The BSC is amethod developed by the Institute for Healthcare Improvement and used for nearly two 
decades across health and social service settings.1? In this section we provide a simplified description of the 
BSC and how it was tested in ECE programs in the CCL project.? 


Following are key elements of the BSC methodology that create the conditions in ECE programs that 
support sustained changes in practice. 


4 For additional technical details about the BSC, refer to other products from the CCL project including the Final Report 
(https://www.childtrends.org/project/culture-of-continuous-learning-project) 
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AQ Engage ECE programs in a learning collaborative 


Staff in ECE programs in one pilot community learned about the project during active outreach activities by 
the CCL project team. Programs completed anapplication stating their interest and capacity to participate. 
The project team developed selection criteria and invited all seven program applicants to join the BSC. This 
engagement supported intentional decision making by the ECE program and ensured that the project team 
was equipped to meet the needs of each program joining the collaborative. Once enrolled, programs 
participated for 12 months. Participants received continuing education credits for time spent in project 
activities. 


Develop shared goals for change 


eee 
HI 


In consultation with local ECE experts, the CCL project team identified quality improvement goals and 
evidence-based practices that guided the BSC. The project team selected goals related to supporting 
children’s social and emotional learning using the Pyramid Model as a guide.2° Other options for early 
childhood content could include health and safety practices, trauma-informed care, or children’s language 
and literacy. In a BSC, each selected goal is paired with metrics to show progress over time. Two monthly 
metrics were selected for the project: behaviors perceived as challenging and two-way communication with 
families. 


e °@ 7 
Gy Learnin teams 


Teams are anessential feature of a BSC. Each of the seven ECE programs in the BSC identified a team that 
included teachers, directors, and parents. These teams interacted within their program and with teams from 
other ECE programs in the BSC. The teams met for four full-day, in-person sessions and participated in 
phone calls and online forums that promoted collaboration and cross-team sharing. Teams varied in size 
from four to eight members. Learning in teams supports participants in developing relationships that help 
them think through challenges, share successful strategies, and feel validated and respected in their work. 
Team work also provides peer accountability that inspires continued progress. The permission granted by 
the center leader for BSC teams to test changes promotes distributed leadership and asense of ownership 
and responsibility for the improvement process. 


| yp Empower and equip participants to test new practices 


The teams identified small changes in social and emotional learning practices to test. They learned new tools 
for collecting data and reflecting on if the change they tried was an improvement. Each team member was 
encouraged to contribute ideas for adapting evidence-based practices for their local context. Successful 
practices were shared with non-participating staff in the program and across programs in the BSC. 


av] Provide expert support on content and the process of improvement 


The BSC included experts on social and emotional learning and on the BSC process for supporting change. 
The experts helped the teams identify evidence-based social emotional practices to try in their classrooms 
and ways to determine if their practices are working. In a BSC, experts arereferred to as faculty coaches. 
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Initial Findings: Adapting a BSC to Early Care and 
Education Settings 


We examined the feasibility of implementing aBSC in center-based child care and Head Start programs by 
asking the following questions: 


e Towhat extent did programs engage with the BSC and participate in the activities offered to 
teams? 

e How successfully did programs develop aculture of continuous learning? 

e Was there initial evidence that participating in the BSC resulted in desired outcomes related to 
teaching practices and organizational culture? 


We conducted interviews, focus groups, and surveys with participants and the team that implemented the 
BSC. In this section, we highlight findings from the discussions of each question and share examples of key 
themes that emerged from the discussions. By design, we did not statistically test observed differences 
because of the small number of teams that participated. 


Q Findings: Programs varied in their capacity to engage in the BSC. 
Participants reported that they appreciated the activities and found them 
different from other quality improvement initiatives. 


A key step in ECE quality improvement is meaningful engagement of programs in improvement activities. 
Given the investment of time needed to participate in the BSC, it was important to assess the interest and 
capacity of programs to commit to the process. 


Programs applied to participate in the project, but the number of applicants was smaller than expected 
given the number of eligible programs. 


e Potential applicants indicated interest, but they were concerned about how staff shortages and 
participation in other projects would impact their capacity to participate fully. 
e The focus on social and emotional learning was a draw for applicants. 


Program participation in activities varied across programs and over time. 


e Some programs had consistent team representation at in-person activities, while others sent only 
one or two representatives or did not attend at all. 

e Participation in in-person activities decreased over time. 

e Senior leaders on teams had competing responsibilities that made it difficult for some to participate. 

e Indicators of program participation were compiled to identify two groups of programs: two 
programs with “robust” participation and five programs with “moderate” participation. These 
groups of participants differed on other organizational characteristics, with robust participants 
appearing to have greater capacity for engagement at the outset of the project. 


Programs completed the data collection on the monthly metrics. 


e Allprograms were able to collect and submit monthly metrics data, but this process was more 
challenging for some to complete on aregular basis. For example, two programs submitted only one 
month of data on children’s negative behavior. 

e Identifying a staff person to lead the metrics collection facilitated the process. 
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Participants enjoyed the BSC and found it to be different from other quality improvement initiatives. 


e Participants reported positive perceptions of the activities in the BSC. 

e Participants valued the in-person meetings and the opportunities to interact with other team 
members through monthly calls. 

e Participants liked the team and the faculty coaching on social and emotional learning practices. 


Findings: The BSC teams invested in developing a culture of continuous 
learning in their programs. 


A culture of continuous learning is created when relationships and structures among teachers, directors, 
and other administrative staff in a program support curiosity, questions, and reflections about current 
practices. They are interested in learning from failures as well as successes and in learning from one 
another. The BSC focused on sparking and amplifying these mindsets and relationships. 


We found preliminary evidence of a culture of continuous learning in programs on multiple indicators. 


e Insome programs, teachers and directors entered the project with mindsets and organizational 
practices that support continuous learning. Measurement of these constructs showed little change 
over time. This orientation toward learning and new ideas may in fact have motivated programs to 
apply to participate in the project. 

e Teachers and other staff reported that they were encouraged by their BSC team to take initiative 
and demonstrate leadership in trying and promoting new practices, regardless of their formal title 
in the program. 

e The teams interacted with other programs to share learning and identify opportunities to test new 
practices. Working with teams in other programs exposed participants to new ideas they may not 
have considered, as well as opportunities to learn from what worked well and what did not work 
well in other programs.2! 


Some personal characteristics (e.g., personal efficacy, sense of psychological safety) associated with 
developing aculture of continuous learning appeared to differ across the robust and moderate 
participation groups. 


e Directors’ and teachers’ beliefs in their ability to make decisions that result in positive outcomes 
(“efficacy”) are part of aculture of continuous learning. Directors and teachers in the robust 
participation group started high and stayed high on measures of efficacy. Directors and teachers in 
the moderate participation group started lower on measures of efficacy, but scores increased over 
time. 

e Psychological safety refers to asense of security and trust that expressing ideas is welcome in the 
program. Measures of psychological safety did not change over time; however, directors and 
teachers in the robust participation group started and stayed higher on the measure than those in 
the moderate participation group. 


Findings: The project saw preliminary changes in teaching practices and 


organizational culture. 

A key question when assessing the feasibility of a quality improvement method in ECE programs is, “Does 
the method have promise to change the outcomes of interest?” The BSC targeted (a) beliefs and practices 
to support children’s social and emotional learning, (b) shared learning with non-team members, (c) 


knowledge and skills related to the process of making improvement, (d) features of workplace culture that 
support continuous learning, and (e) sustained changes after participating in the BSC. 
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We saw preliminary evidence of changes across most but not all the outcomes of interest. As noted, 
statistical tests were not conducted on the observed outcomes. 


Beliefs about social and emotional learning 


e Teammembers demonstrated little change over time on ameasure of beliefs about social and 
emotional learning. We did not see differences in scores on the measure between the robust or 
moderate participation groups. Programs with a strong orientation to social and emotional learning 
may have been more inclined to apply for the CCL project than other programs. 


Shared learning 


e When program teams identified successful practice changes, they shared information with other 
teams. Teams adopted new practices related to, for example, family engagement strategies, options 
for recognizing staff contributions, and techniques for supporting staff well-being under stress. 


e Some program team members reported challenges in sharing the tools from the BSC with non-team 
members. Other team members provided examples of how they shared learning with non-team 
members in their programs. 


Knowledge of strategies to support continuous improvement 


e Teams reported they had acquired new knowledge about making and sustaining improvements 
using the tools and skills obtained through the learning collaborative. 


e Teammembers described how becoming familiar with data and data collection procedures helped 
them become more confident in their work and recognize that improvement is an ongoing process. 


Leadership and organizational culture 


e Teammembers reported an appreciation for seeing members of their program demonstrate 
leadership, regardless of the person’s job title. 


e Job satisfaction appeared to increase over time across teams with either robust or moderate 
participation, but different patterns were observed. Members of teams with robust participation 
were higher in measures of satisfaction at both time points than those with moderate participation 
on all the dimensions that were measured - working conditions, co-worker relations, the work 
itself, and supervisor relations. Those on robust participation teams reported increases over time 
on satisfaction with working conditions and supervisor relations. Those on moderate participation 
teams reported increases over time on satisfaction with co-worker relations and the work itself. 


Sustained changes 


e Teamsreported some continuation of quality improvement practices (such as collecting metrics) six 
months after participation in the BSC ended. 


After examining BSC processes and outcomes, initial evidence indicates that implementing a BSC in ECE 
programs is feasible and a promising strategy for addressing challenges in current quality improvement 
initiatives. Though further testing of the methodology is necessary, the findings from this pilot provide 
decision makers with information about how challenges in ECE quality improvement can be addressed using 
amethod from the health sector. The following section offers considerations for implementing a BSC in ECE 
programs. 
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Implications of the Findings for Quality Improvement 
Initiatives in Early Care and Education Systems 


The Breakthrough Series Collaborative is a quality improvement methodology compatible with the needs 
and priorities of ECE programs and has the potential to fit within ECE quality improvement systems. The 
CCL case study offers ECE leaders in child care and Head Start a rich example and set of considerations for 
why and how a BSC might fit in a menu of options offered to ECE programs. 


A key feature of the BSC is the focus on supporting evidence-based teaching practices while also addressing 
the organizational processes and relationships that promote sustained change. The BSC can support 
teachers in applying what they know to what they do through supportive structures that promote positive 
relationships and a sense of trust and shared goals among team members. The method can be used across 
different content areas and teaching practices which makes it a versatile option for the ECE system. 


The focus of the BSC on elevating the voices of teachers and supporting them as change agents in their 
programs also aligns with a focus in ECE on equitable practices. To fully address the needs of teachers and 
program staff, future piloting of the BSC could be combined with efforts to address wage disparities and 
economic insecurity among the ECE workforce. 


The CCL findings indicate that even with the enthusiasm of participants, it is important to consider how 
programs with varying levels of capacity can engage in the BSC. ECE leaders can identify the additional 
supports or adaptations needed to help programs with fewer initial resources or capacity to engage in this 
type of quality improvement process. 


Using data to track progress on monthly metrics is an activity that fits well with existing ECE practices 
though training and support would be needed to ensure that programs have the tools they need to be 
successful. The CCL project revealed challenges in collecting data, especially on a regular schedule. 
Programs would also need resources to address staffing and substitutes to allow them to participate in BSC 
activities in an ongoing way. 


The CCL study also raises questions that can be addressed in future BSC implementation studies. Future 
work can address the extent to which a BSC can be embedded in QRIS and Head Start TAsystems and how 
it compares in cost and effectiveness in promoting quality improvements. The CCL study findings suggest 
that certain program characteristics may be a better fit with BSC activities. An implementation study can 
assess options for identifying program participants and understanding how their characteristics impact 
their participation and ultimate success in a BSC. 
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